The Internet is a source of much useful information. However, the content of the internet is also polluted with spam data. Spam filtering is therefore crucial as it helps to reduce the level of noise contained in information obtained by web crawling. A “spam flood” refers to a phenomena where hundreds or thousands of spam web pages are introduced into the Internet over the span of a few hours (in turn, a spam flood can propagate into a web crawler system and cause unwanted responses).
Accordingly, it is desirable to have a computer implemented methodology for detecting the occurrence of a spam flood that involves spam web pages, blog posts, or the like. In addition, it is desirable to provide and maintain a system that is capable of dynamically responding to a spam flood in an efficient and effective manner. Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.